Smart Phrase Generator to Instruct Digital Manikin Action

ABSTRACT

Embodiments determine manikin posture for simulations of real-world environments. An embodiment automatically generates a phrase by performing a hierarchical analysis using data regarding the real-world environment. According to an embodiment, the generated phrase describes a task, to be simulated, performed by a manikin in an environment. In turn, one or more posture engine inputs are determined based on the generated phrase. The posture for the manikin in a simulation of the manikin performing the task in the environment is then automatically determined based on the determined one or more posture engine inputs.

BACKGROUND

A number of existing product and simulation systems are offered on themarket for the design and simulation of objects, e.g., humans, parts,and assemblies of parts, amongst other examples. Such systems typicallyemploy computer aided design (CAD) and/or computer aided engineering(CAE) programs. These systems allow a user to construct, manipulate, andsimulate complex three-dimensional (3D) models of objects or assembliesof objects. These CAD and CAE systems, thus, provide a representation ofmodeled objects using edges, lines, faces, polygons, or closed volumes.Lines, edges, faces, polygons, and closed volumes may be represented invarious manners, e.g., non-uniform rational basis-splines (NURBS).

CAD systems manage parts or assemblies of parts of modeled objects,which are mainly specifications of geometry. In particular, CAD filescontain specifications, from which geometry is generated. From geometry,a representation is generated. Specifications, geometries, andrepresentations may be stored in a single CAD file or multiple CADfiles. CAD systems include graphic tools for representing the modeledobjects to designers; these tools are dedicated to the display ofcomplex objects. For example, an assembly may contain thousands ofparts. A CAD system can be used to manage models of objects, which arestored in electronic files.

CAD and CAE systems use of a variety of CAD and CAE models to representobjects. These models may be programmed in such a way that the model hasthe properties (e.g., physical, material, or other physics based) of theunderlying real-world object or objects that the model represents.Moreover, CAD/CAE models may be used to perform simulations of thereal-word objects/environments that the models represent.

SUMMARY

Analyzing ergonomics and simulating an agent in an environment arecommon simulation tasks implemented and performed by CAD and CAEsystems. Here, an agent refers to an entity which can observe and actupon an environment e.g., a human, an animal, or a robot, amongst otherexamples. Such functionality can be used to automatically predictbehavior of the agent in the environment when performing a task with oneor more target objects. Amongst other examples, these simulations candetermine position and orientation of a human when assembling a car in afactory. The results of the simulations can, in turn, be used to improvethe physical environment. For example, simulation results may indicatethat ergonomics or manufacturing efficiency can be improved byrelocating objects in the environment.

Performing simulations requires a posture, i.e., position andorientation, of the agent, i.e., digital human model, manikin, etc., ina virtual representation of the environment of interest. In commondigital human modelling, posturing a manikin requires manipulation ofjoint degrees of freedom or kinematic chain end-effector manipulation.In the last decade, manikin posturing has evolved and can be implementedusing manually defined targets for each kinematic chain end effector,e.g., hands and vision, however time is still considered to be a bigobstacle to the use of Digital Human Models [8]. Further, manikinposturing in virtual environments can be implemented using direct andindirect kinematics and/or manually defined targets. Even with thelatest capabilities, from Jack [1] and IPS IMMA [2] to name just a few,manual intervention that consumes manufacturing engineers' precious timeis still required to determine manikin posture [3].

Embodiments solve these problems and provide functionality that does notrequire laborious manual intervention to determine a posture for amanikin in a simulation of a real-world environment. An examplecomputer-implemented embodiment begins by receiving environment data.Next, such an embodiment automatically generates a phrase by performinga hierarchical analysis using the received environment data. In anembodiment, the generated phrase describes a task performed by a manikinin an environment that is going to be simulated. To continue, one ormore posture engine inputs are determined based on the generated phrase.In turn, a posture for the manikin in a simulation of the manikinperforming the task in the represented environment is automaticallydetermined based on the determined one or more posture engine inputs.That is, a digital processor automatically (without human intervention)computes or otherwise configures posture for the manikin as a functionof the determined one or more posture engine inputs.

According to an example embodiment, the environment data includes atleast one of: one or more operations; a sequence of tasks comprising anoperation; one or more parts assigned to an operation; one or moreresources assigned to an operation; a sequence of operations; one ormore objects in the environment; position of the one or more objects inthe environment; context of the task; and task validation premises.

In an embodiment, the generated phrase is comprised of: (i) a verbcomponent indicating an action of the task (e.g., an operation), (ii) awhat component (e.g., subject component, noun component, operator,operand, etc.) indicating what the verb component pertains to, (iii) apreposition component, and (iv) a with or where component indicating anobject used for the action or where the action is performed.

An example embodiment performs the hierarchical analysis using thereceived environment data by first determining candidate phrases basedon the environment data and, second, hierarchically filtering thecandidate phrases based on the environment data to generate the phrase.According to an embodiment, hierarchically filtering the candidatephrases comprises eliminating invalid phrases from the determinedcandidate phrases and eliminating phrases with an invalid interactionobject.

Another embodiment, prior to determining the one or more posture engineinputs, receives an indication of user approval of the phrase. Further,according to another embodiment, the one or more posture engine inputsinclude at least one of: grasp target, vision target, vision acuity forthe manikin performing the task, and an indication if object weight isconsidered. Another embodiment determines the one or more posture engineinputs based on the generated phrase by searching a mapping usingcomponents of the generated phrase, wherein results of the searchingindicate the one or more posture engine inputs. In an exampleembodiment, the mapping indicates respective posture engine inputscorresponding to respective phrase components.

According to an embodiment, automatically determining the posturecomprises: (i) providing the determined posture engine inputs to aposturing engine configured to determine (generate) the posture basedupon the provided posture engine inputs, and (ii) receiving thegenerated posture from the posture engine.

An embodiment simulates the manikin performing the task in therepresented environment using the determined posture. Such an embodimentmay determine a change to a virtual workstation for the manikinperforming the task based on results of the simulation which in turnprovides an indication of change to the corresponding real-worldworkstation for performance of the subject task.

Another embodiment of the present invention is directed to a system thatincludes a processor and a memory with computer code instructions storedthereon. In such an embodiment, the processor and the memory, with thecomputer code instructions, are configured to cause the system toimplement any embodiments or combination of embodiments describedherein.

An embodiment is directed to a cloud computing implementation fordetermining manikin posture. Such an embodiment is directed to acomputer program product executed by a server in communication across anetwork with one or more clients. The computer program product comprisesprogram instructions which, when executed by a processor, causes theprocessor to implement any embodiments or combination of embodimentsdescribed herein.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing will be apparent from the following more particulardescription of example embodiments, as illustrated in the accompanyingdrawings in which like reference characters refer to the same partsthroughout the different views. The drawings are not necessarily toscale, emphasis instead being placed upon illustrating embodiments.

FIG. 1 is a flowchart of a method for determining a posture for amanikin according to an embodiment.

FIG. 2 illustrates a digital environment in which an embodiment may beimplemented.

FIG. 3 is a user interface showing a phrase generated by an embodiment.

FIG. 4 is a simplified block diagram of a system embodiment forgenerating a phrase for posture determination.

FIG. 5 is a flow diagram of a phrase filtering method that may beimplemented by embodiments.

FIG. 6 is a table depicting a prioritization logic that may be employedin embodiments.

FIGS. 7A-G illustrate steps of using an embodiment to posture a manikinin a digital environment of interest.

FIG. 8 is a simplified diagram of a computer system for determiningmanikin posture according to an embodiment.

FIG. 9 is a simplified diagram of a computer network environment inwhich an embodiment of the present invention may be implemented.

DETAILED DESCRIPTION

A description of example embodiments follows.

Analyzing ergonomics and simulating an agent in an environment arecommon simulation tasks implemented and performed by CAD and CAEsystems. Performing these simulations requires a posture, i.e., positionand orientation, of the agent, i.e., digital human model, manikin, etc.,in the environment.

Functionality exists, such as the Smart Posturing Engine (SPE) developedby the Applicant, to automatically posture a manikin based upon inputs.However, there is no such functionality to automatically determine theinputs required by posture engines, e.g., SPE. As such, to provide auser-friendly posture solution functionality is needed to determineinputs for posture engines. A good interface to determine inputs forthese existing posturing engines should be comprehensible to the userand gather enough information for determining the posture engine inputs.Embodiments fulfill these requirements and much more. Embodimentsgenerate a phrase which presents in words the desired manikin task in avisual interface. In turn, this generated phrase can be used todetermine posture engine input and ultimately a posture for a manikin.

Embodiments provide an automated new way of instructing a manikin taskso as to compute a probable and repeatable posture. Embodiments utilizemultiple inputs, such as the operations assignations and the surroundingresources in the environment, and then, generate a logic “phrase”describing, in the user language (i.e., natural language), the task tobe simulated. This phrase can then be provided to a posturing engine todetermine a posture for the manikin. Such functionality prevents a userfrom having to manually posture a manikin. If desired, the user canvalidate the phrase with a single click to generate a posture. Thissaves user time and induces uniformity in the results between users,e.g., engineers.

FIG. 1 is a flowchart of a method 100 for determining a posture for amanikin according to an embodiment. The method 100 begins at step 101 byreceiving environment data. The method 100 is used to determine aposture for a manikin when performing a task in an environment that isbeing simulated. Thus, the environment data received at step 101 may beany data regarding the environment and task being simulated. Forinstance, according to an example embodiment, the environment dataincludes at least one of: one or more operations; a sequence of taskscomprising an operation; one or more parts assigned to an operation; oneor more resources assigned to an operation; a sequence of operations;one or more objects in the environment; position of the one or moreobjects in the environment; context of the task; and task validationpremises, amongst other examples. It is noted that the foregoing listingis but an example of environment data and embodiments are not limited toutilizing the listed data. Further, there may be variations amongst thelisted data. For instance, an operation may not have any assigned partand may not have any assigned resource.

The method 100 is computer implemented and, as such, the environmentdata may be received at step 101 from any memory or data storage thatcan be communicatively coupled to a computing device implementing themethod 100.

To continue, at step 102, the method 100 automatically generates aphrase by performing a hierarchical analysis using the receivedenvironment data. The generated 102 phrase describes a task, to besimulated, performed by a manikin in an environment. According to anembodiment, the phrase generated at step 102 is a natural languagephrase describing the task to be performed by the manikin in theenvironment. For instance, in an example embodiment, the generatedphrase is comprised of: (i) a verb component indicating an action of thetask, (ii) a what component indicating what the verb component pertainsto, (iii) a preposition component, and (iv) a with or where componentindicating an object used for the action or where the action isperformed.

In an embodiment of the method 100, the hierarchical analysis performedat step 102 includes the filtering method 550 described herein below inrelation to FIG. 5 . According to another embodiment, at step 102,performing the hierarchical analysis using the received 101 environmentdata includes, first, determining candidate phrases based on theenvironment data. An embodiment of the method 100 utilizes a databasestoring possible phrases. In such an embodiment, the phrases in thisdatabase are the candidate phrases. Second, the determined candidatephrases are hierarchically filtered based on the environment data togenerate the phrase. Amongst other examples, hierarchically filteringthe candidate phrases includes eliminating invalid phrases from thedetermined candidate phrases and eliminating phrases with an invalidinteraction object.

To illustrate, consider an example where, at step 101, environment datais received that indicates that the task being simulated is tightening abolt with a wrench. At step 102, the identified candidate phrases arethe phrases in a database (a database of all possible phrases). In suchan illustrative embodiment, first, invalid phrases are eliminated fromthe identified candidate phrases. In this example, all phrases fromamongst the candidate phrases that do not involve tightening a bolt areeliminated at the first stage of the hierarchical analysis. Second, fromamongst the remaining candidate phrases, all phrases, with an invalidinteraction object are eliminated from the remaining candidate phrases.At this second stage, in this illustrative example, all phrases that donot involve tightening with a wrench are eliminated, e.g., phrases thatinclude an impact driver are eliminated.

Returning to FIG. 1 , at step 103, one or more posture engine inputs aredetermined based on the generated phrase output of step 102. Accordingto embodiments, the one or more posture engine inputs determined at step103 include at least one of: grasp target, vision target, vision acuityfor the manikin performing the task, and an indication if object weightis considered.

An embodiment determines the one or more posture engine inputs at step103 based on the generated phrase by searching a mapping usingcomponents of the generated phrase. In such an embodiment, results ofthe searching indicate the one or more posture engine inputs. Themapping utilized in such an embodiment indicates respective postureengine inputs corresponding to respective phrase components.

To illustrate step 103, consider an example embodiment where thegenerated phrase (output from step 102) is “tightening a bolt with awrench.” The mapping to this phrase would return, as the posture engineinputs: the first item (tightening) as a precision view required, thesecond item (the bolt) as the vision target, and the third item (thewrench) as the grasping target.

An embodiment of the method 100 requires user approval of the phrasegenerated at step 102 prior to determining the one or more postureengine inputs at step 103. To facilitate this approval, such anembodiment provides the phrase generated at step 102 to a user. Forexample, the phrase generated at step 102 may be presented to a user ina graphical user interface. The user then provides some indication ofphrase approval and this indication of approval is received at thecomputing device implementing the method 100 and, in response, themethod 100 moves to step 103.

At step 104, a posture for the manikin in a simulation of the manikinperforming the task in the environment is automatically determined basedon the determined one or more posture engine inputs. According to anembodiment, the posture determined at step 104 is a position andorientation for the manikin in the environment. For example, the posturemay be an initial position and orientation for the manikin whenperforming the task being simulated. In an embodiment, the posture forthe manikin is determined at step 104 by (1) providing the determinedposture engine inputs to a posturing engine configured to determine (orotherwise generate) the posture based upon the provided posture engineinputs and (2) receiving the generated posture from the posture engine.

An embodiment of the method 100 goes further and simulates the manikinperforming the task in the environment using the posture determined atstep 104. The results of performing the simulation may be used to modifythe real-world environment that was simulated. For example, thesimulation results may indicate that a change to a workstation positionthe human is using would improve ergonomics and efficiency.

What follows is a further description and examples of embodiments andthe functionality thereof in the process engineering domain in amanufacturing context. However, it is noted that embodiments are notlimited to the process engineering domain and manufacturing context, andembodiments may be utilized for any application in any domain, industry,or context where manikin posture determination is desired.

In a manufacturing simulation context, as an overview, an exampleembodiment, first predicts the most probable working tasks to beexecuted by the manikin. One such embodiment dynamically generates thisprediction based on (i) a process plan, (ii) surrounding resources inthe environment, and (iii) already existing working tasks. Second, suchan embodiment presents the predicted most probable working task(s) tothe user in a common language sentence. Third, the user validates thephrase and, in response, the posture for the manikin is automaticallygenerated by the embodiment method and system.

Process engineering defines and deals with notions like manufacturingbill of materials (MBOM), resources, systems, and operations. Thus, thefollowing information explains how embodiments use the processengineering information (e.g., environment data) to predict the mostprobable phrase for purposes of generating a manikin posture.

Generally, engineering process systems contain some listing of theoperations to be executed by the worker, i.e., manikin, in thesimulation. However, an operation can represent either a generalstatement or a very detailed small step. To allow the detailing of anyoperation, especially the general ones, a system executing embodimentsallows operations to be further refined. The refined operations are madeof worker tasks (WT) and they may be structured in the listing ofoperations as the children of operations. In an embodiment, eachoperation may be divided in several WTs, where each WT represents one ofthe postures a worker would take. In other words, each WT represents apoint in the simulation for which a posture determination is desired andembodiments may be utilized to determine such postures.

FIG. 2 illustrates a simulation program interface 220 where embodimentsmay be used to determine postures for the manikin 221 in the environment222. The interface 220 includes the navigator 223 that shows anorganization of systems 224, operations 225 a-d, and WTs 226 a-e.

In the interface 220 each WT 226 a-e utilizes a posture and presents apoint in the simulation of the manikin 221 operating in the environment222 where a phrase may be generated using embodiments to determine aposture. In an implementation, a phrase is automatically generated by anembodiment for each WT 226 a-e and, consequently, a posture is generatedby a posturing engine, e.g., SPE, using the generated phrases. Inembodiments, the generated phrases are not default phrases and, instead,the generated phrases are particular to the simulation and the tasks 226a-e being simulated.

Input Information

Embodiments use various information, e.g., environment data, to generatephrases. Example information includes information about objects in theenvironment (e.g., CAD modeled objects), manufacturing premises, andinformation that is available from the simulation system, e.g., anengineering process planning system.

One example of data used in embodiments is data stored in the simulationsystem, e.g., process planning system, in which the posture is going tobe used. In the simulation planning phase, a process engineer normallydefines the operations to be simulated. Defining the operations mayinclude assigning, to the operations, part(s) that will be added to theassembly (from a manufacturing bill of materials) and assigning whatresource(s) will be used (e.g., tooling). This allows embodiments toprovide a visualization of the product buildup (e.g., an indication ofthe assembly state of a product at a specific time on a production line)when navigating through the operations. In addition, a process engineercan also define a position to those assigned items (parts and resources)to visualize where the assembly task occurs, e.g., on an assembly line.This information is very useful to embodiments, but insufficient todetermine the task the worker has to do for each operation. Forinstance, the location of manikin hands and vision targets are unknowneven when the location of objects is known.

Other information that can be utilized by embodiments is sequenceinformation. For instance, if embodiments are being utilized as part ofa simulation that includes several different operations, the order ofthese operations can be used by embodiments to determine the phrase. Inone such embodiment, an operations system (e.g., DELMIA Process Planningprovided by the Applicant) contains information on the sequence. Anembodiment can access flow links defined in a process planningapplication and, by accessing the flow links, an embodiment discernswhich operation happens before or after each operation.

An embodiment processes known environment data (e.g., assigned items tothe operations) to identify surrounding object(s) which may be ofinterest. One such embodiment utilizes a volumetric filter with theknown objects (product part or resources) location information toidentify the location of additional objects. To illustrate, in anembodiment where the task being simulated is on an assembly line, anembodiment can identify objects in the environment using a volumetricfilter. Such functionality can be utilized to identify resources thatmay be required by the operation (thus part of the “Phrase”), but wherethe resources have not been assigned.

Yet another example of environment data that may be utilized inembodiments is operation context data. The context when a worker task iscreated results in data that may be used by embodiments. For example, ifthe user moved an object, this context indicates it is highly probablethat this moved object should be included in the predicted phrase.

Another embodiment also considers validation information. For example,in order to choose the most probable phrase, an embodiment heavilyweights manufacturing task validation. In such an example, the inputdata identifies whether, for instance, necessary parts are available.This allows phrase generation to prioritize an assembly task over partsupply from storage. This is because if the assembly task is notpossible, then it may be useless, at this point in time, to validate thesupply logistic tasks.

Phrase Generation

While a user can manually set worker task phrases, avoiding manuallysetting phrases through use of embodiments saves a considerable amountof user time. Advantageously, embodiments perform automatic reasoning onthe input data, e.g., environment data, to automatically set the phraseaccording to the simulation being performed, e.g., manufacturing processcontext. According to an embodiment, the phrase is composed of fouritems (i) an “action” verb, (ii) a “what”, (iii) a “preposition,” and(iv) a “with or where.” FIG. 3 shows an example phrase 331 in theinterface 330 that would lead to a worker screwing a bolt holding ascrewdriver in his right hand. In embodiments, the predicted phrase canevolve as data is being obtained, e.g., as WTs are created. From theuser perspective, the result of embodiments is not only to define thedesired phrase, but also to enable worker task creation in one click.

FIG. 4 illustrates a system 440 that is configured to generate a postureaccording to an embodiment. The system 440 begins with the contextualinputs that include the systems/operations inputs 441 and the 3D inputs443. The systems/operations inputs 441 include the assignations 442 aand sequences 442 b. The assignations 442 a are the declaration, usuallydone by someone with the role of the process planner, of what part(s)(e.g. a bolt) and/or resource(s) (e.g. a wrench) are provided to aspecific operation. That person also defines the sequences 442 b bydeclaring which operations happen before or after other operations(e.g., insert washer operation occurs before a nut tighteningoperation). The 3D inputs 443 include the surrounding object data 444 a(i.e., indications of known objects (e.g., objects and types) within aspecific range around the assigned 442 a objects) and active selectionindication 444 b (i.e. the last 3D object the user has voluntarilyselected in the interface, e.g., 220). The system 440 also includes thephrase generator 445 and the smart posturing engine 446 that outputs aposture 447. In the system 440 the phrase generator 445 generates aphrase and derives the inputs needed by the posturing engine 446 fromthe generated phrase.

In operation, the phrase generator 445 operates in conjunction with adatabase of “phrase parts.” Such a database is structured to include alisting of possible phrase components. For instance, if phrases includean (i) “action” verb component, (ii) a “what” component, (iii) a“preposition” component, and (iv) a “with or where” component, such adatabase would include a list of all possible verb, what, preposition,and with or where components that may be used to construct phrases.

The phrase generator 445 first filters the database using the input data442 a-b and 444 a-b to determine any possible combinations of logicaland valid phrases composed of an action 450, what 451, and with or where452. In other words, this first step identifies possible action 450,what 451, and with or where 452 components in the database that are inaccordance with the data 442 a-b and 444 a-b. At this point, there aremultiple phrases possible. For example, if a part, a screwdriver, and ajig has been assigned to an operation, the phrases “Screw part with thescrewdriver” and “Place the part on a jig” are both probable and valid.In turn, the phrase generator 445 applies a series of decision steps andfilters to narrow down the possibilities and end with a final phrase.This series of decision steps and filters may be carried out using themethod 550 described herein below in relation to FIG. 5 . In operation,after the possible combinations of valid phrases are identified, themanufacturing actions 449 and manufacturing premises 448 are consideredto further refine the listing of possible action 450, what 451, and withor where 452 components to the most probable action 450, what 451, andwith or where 452 components. Examples of manufacturing premises 448include (i) prioritizing a candidate item in a phrase listing if it is ausable resource (i.e., the manikin can interact with it) that has beenassigned to the operation (this is because a user explicitly declaredits use) and (ii) not all resources have equal value for purposes ofinteraction (e.g., a hand tool should be prioritized over storage, whichshould be prioritized over furniture). According to an embodiment,manufacturing actions 449 are actions that add to a product. Examplemanufacturing actions 449 include assembling, clipping, connecting,riveting, screwing, and tightening, amongst other examples. In anembodiment, manufacturing actions 449 do not include actions such asgetting, touching, checking, and unloading, amongst other examples.

In an embodiment, the action component 450 is identified using themanufacturing action indication 449 and sequence indication 442 b. Thewhat component 451 is identified using the assignation 442 a, sequence442 b, surrounding object 444 a, and active selection 444 b. Likewise,the phrase generator 445 determines the with or where 452 using theassignation 442 a, sequence 442 b, surrounding object 444 a, and activeselection 444 b. The action component 450, what component 451, and withor where component 452 make up the phrase.

To continue, the phrase generator 445 determines the posture engineinputs, which include the grasp target 453, vision target 454, andvision acuity 455. In an embodiment, the phrase generator 445 uses amapping that indicates what the grasp target 453, vision target 454, andvision acuity target 455 should be based on the determined actioncomponent 450, what component 451, and with or where component 452. Inother words, this mapping translates the determined phrase components,action 450, what 451, and with or where 452, to the grasp target 453,vision target 454, and vision acuity 455. For example, with the phrase“Align part with pliers”, the mapping indicates the pliers is the grasptarget and the mapping indicates that since the object to be aligned isnot the pliers, the part is the vision target. Also since the actionrequires landmark or details, the mapping indicates the vision acuity is“precision” rather than only “within field of view.”

To continue, the grasp target 453, vision target 454, and vision acuity455 are provided to the posturing engine 446 that is configured todetermine the posture 447 (i.e., a position, location, and orientationof the manikin while reaching the target, along with the hand and visionof manikin, without collision) based on the provided grasp target 453,vision target 454, and vision acuity 455. With these inputs 453, 454,and 455, the posturing engine prepositions the digital human in frontand towards the object(s) to grab, estimates the grasp mode and grasplocation, and finally estimates the whole manikin posture. In anembodiment, the posturing process performed by the engine 446 relies onan optimization solver based on the pseudo-inverse of the Jacobianmatrix that involves postural stability, joint mobility (including lineof sight) and ergonomic guidelines (e.g. ISO11226, EN1005-3).

In an example embodiment, after removing invalid phrases, the phrasegenerator 445 determines if other worker tasks already exist in theoperation where the user is creating the new one, so as to orient theappropriate methodology to use. An embodiment check if a worker taskand/or phrase already exists for an operation. For instance, if noworker task already exists within an operation, all systems andoperations 441 as well as 3D inputs 443 are important to consider. Whenthis occurs, the phrase generator 445 may use the flow 550 describedherein below in relation to FIG. 5 . However, when a worker task alreadyexists, the phrase to be determined is more likely related to theexisting worker task (before and/or after), and, thus, in such anembodiment, the phrase generator 445 uses a mapping between a generatedphrase and phrases that should be before and/or after the generatedphrase. An example of this mapping between a generated phrase andphrases that are before and after is shown in FIG. 7G in the table 782of all possible phrases. This decision relies on the manufacturingpremises input 448. Manufacturing engineers use strategies to validatethe manufacturability of the product. Manufacturing engineers typicallystart validation where it is more valuable to spend time. For example,such engineers virtually validate the main tasks (usually assemblytasks) before validating storage location tasks. Doing this avoidsworking on accessory tasks at risk to change due to the product, thesequence, or a resource changing. Indeed, if an assembly task needs tobe changed, there is a high chance that the step just before and afterwill also require change. Thus, an embodiment uses the main working taskidentified in the manufacturing premise 448, when existing, and uses abefore/after logic to define the possible actions 450 to use in thephrase. In an embodiment, action 450 refinement is implemented usingoperation assignation logic (e.g., decision flow 550) and results in amanufacturing transformation action like screw, drill, etc. beingidentified. Otherwise, the driving input is the operation assignationlogic (e.g., to complete a transformation task, accessory tasks likegetting and releasing object are required).

When an operation already has a phrase (in the format of a WT), anembodiment provides a user interface that allows the user to specifywhere any new task should be added in the sequence, e.g., before orafter another WT. In such an embodiment, the phrase determination logiccan predict what action to use following an established action order.For example, getting a tool precedes its use and assembling a partfollows aligning the part.

Once the action 450 is defined, another logical flow is implemented todetermine the what 451. Determining the what 451 considers the assigned442 a and surrounding objects 444 a to adapt the phrase to the context.For example, place a tool back to storage after using it. In some cases,the logic may repeat a task (i.e., same phrase, but using a differentpart, for example screwing a number of screws).

According to an embodiment, when the phrase is the first of anoperation, the phrase determination logic focuses on the presence of theassignations 442 a first (as What 451 or With or Where 452) rather thanthe action 450 to perform. This hierarchy respects user intention. Forinstance, if a user took the time when creating an operation to specifywhat objects are involved, these assigned 442 a objects are consideredfirst.

The phrase generator 445 may be configured to prioritize some types ofobjects over others depending on the manufacturing purpose. Forinstance, not all object types are equal and some objects are consideredaccessories to a manufacturing task. These details may be coded in thelogic implemented by the phrase generator 445 depending on themanufacturing operations being simulated.

Further, the phrase generator 445 may consider an object to be amandatory part of the phrase. In a case where the user initiates theworker task creation by positioning an object in the 3D environment, thephrase generator 445 may require this object, which is an activeselection 444 b, to be in the phrase. In such a case, in replacement ofthe first filter (in an embodiment where the phrase generator 445 isimplementing the method 550), all phrases that are not involving theobject positioned by the user, are removed.

As noted above, in an embodiment the phrase generator 445 starts with alist of all valid phrases. Then, the phrase generator 445 appliesfilters to that list of valid phrases to determine the phrase. FIG. 5illustrates an example filtering, i.e., hierarchical analysis, method550 that may be applied to a listing of valid phrase to determine thephrase. Amongst other examples, the method 550 may be utilized at step102 of the method 100 described hereinabove in relation to FIG. 1 and inthe system 440 by the phrase generator 445 described hereinabove inrelation to FIG. 4 .

The method 550 starts 551 and at step 552 considers if a resource (e.g.tooling) is assigned. Such functionality considers at step 552 if anobject, i.e., resource, is assigned to the operation hosting the workertask for which a phrase is being determined. According to an embodimentin a product manufacturing context, a resource is any object that is notthe object/product being manufactured. Further, in an embodiment, anindication of a resource or data related thereto, may be stored in adistinct data structure from data related regarding the objecting beingdeveloped. If step 552 determines that a resource is assigned to theoperation, the method 550 moves to step 553 and otherwise, the method550 moves to step 554.

At step 554, the method 550 considers if there is a previous assemblyinvolved in the operation and if there is a previous assembly, themethod 550 moves to step 555 and, otherwise, moves to step 556. Step 554considers if, at the operation hosting the worker task to be created, anassembly exists as buildup output of a previous operation. In otherwords, step 554 considers if there is an existing assembly from aprevious operation or task that is related to the current operation forwhich the phrase is being generated. For example, if a previousoperation involved putting a tire on a wheel and the current operationinvolves installing the tire/wheel assembly on a car, the tire and wheelwould have been considered provided objects when a phrase for puttingthe tire on the wheel was determined, and at step 554 when thetire/wheel combination is being installed on the car the method 550would determine that there is a previous assembly, the tire/wheelcombination.

Step 555 considers if a provided object (i.e., a new part) is assignedto the operation. Step 555 thus evaluates if a new object is assignedfor implementing the buildup of the operation hosting the worker task tobe created. Returning to the wheel/tire assembly example, if a nut isneeded to install the tire/wheel assembly on the car, step 555 woulddetermine that an object, the nut, is assigned to the operation. Ifthere is a provided object assigned, the method 550 is in scenario 6,where the task involves an assembly and a provided object. In scenario 6all phrases including a resource or not including an assembly or notincluding a provided are rejected. If a provided object is not assigned,the method 550 is in scenario 2 where only an assembly is involved inthe task. Thus, in scenario 6 all phrases including a resource or notincluding assembly are rejected.

Returning to step 556, the method 550 considers if a provided object isassigned. If, at step 556, the method 550 determines a provided objectis assigned to the operation that includes the worker task to becreated, the method 550 is in scenario 1 where only a provided object isassigned to the task. In scenario 1 all phrases including a resource oran assembly or not including a provided are rejected. If, at step 556,the method 550 determines that a provided object is not assigned to theoperation including the worker task to be created (the task for which aphrase is being determine), the method 550 moves to step 557 which looksfor an object involved in a what-with/where list, i.e., a list ofcandidate objects to be a What 451 or With or Where item 452 constitutedof assigned and surrounding objects). An example what-with/where list781 is shown in FIG. 7F. After looking for the object in thewhat-with/where list at step 557, at step 558, the method 550 considersif an object is listed in the what-with/where list of the worker task tobe created, but is not an assigned resource to the worker task parentoperation. If step 558 determines there is an object listed in thewhat-with/where list, but assigned to the parent operation, the method550 moves to step 559 and if step 558 determines there is not an objectlisted in the what-with/where list, the method 550 is in scenario 8.Scenario 8 is an “Else” case, i.e. scenario 8 identifies a phrase to usewhen none of the other use cases apply. For instance, the phrase forscenario 8 could be “No action,” since there is no assembly, noprovided, and no valid resource assigned or surrounding.

If the method 550 reaches step 559, it means that only unassignedresources (no provided and no assembly) are available for the phrase.Step 559 considers if there is more than one resource available. Ifthere is not more than one resource, the method 550 is in scenario 3.1.In scenario 3.1 the possible phrases are limited to phrases includingthe unique available resource, and phrases that include an assembly andprovided object are filter out, i.e., eliminated. If there is more thanone resource, the method 550 moves to step 560.

Returning to step 552, where it is determined that a resource isassigned to the operation that includes the worker task to be createdand for which the phrase is being determined, the method 550 moves tostep 553. Step 553 considers if a provided object is also assigned tothe operation. If step 553 determines a provided object is assigned, themethod 550 moves to step 561. If step 553 determines a provided objectis not assigned, the method 550 moves to step 562.

At step 561, the method 550 considers if there is no an assembly andalso considers if there is a specific hand tool (e.g., nut runner,reaction bar, rivet runner, screwdriver, power screwdriver, torquewrench, tweezers) assigned to the operation hosting the worker task tobe created. If there is no assembly, but a hand tool is assigned, themethod 550 is in scenario 4.1. Scenario 4.1 is reached in cases wherethe data does not provide a 3D object for fastening/interaction andthere is no assembly, but there is a hand tool. When this occurs, thepossible phrases include the specified resource, e.g., the hand tool,and the with or where 452, or the what 451 elements of the phrase arereplaced by “virtual fasteners.” An example suggested phrase may be“Tighten virtual fastener with torque wrench.” If step 561 determinesthat an assembly is assigned, the method 550 is in scenario 4.2. Inscenario 4.2 all phrases that do not include a resource and a providedobject are rejected.

Returning to step 553, when the method 550 determines a provided objectis not assigned, the method 550 moves to step 562. Step 562 considers ifthere is a previous assembly assigned to the task. If there is aprevious assembly, the method 550 is in scenario 5. In scenario 5 thepossible phrases are limited to phrases including a resource and anassembly, and phrases that include a provided are filtered out. If step562 determines there is not a previous assembly, the method 550 moves tostep 563.

Step 563 considers if there is more than one resource assigned to thetask. If there is not more than one resource, the method 550 is inscenario 3.1. In scenario 3.1 the possible phrases will be limited tophrases including the unique resource, and phrases that include otherresources or an assembly or a provided are eliminated. If step 563determines there is more than one resource, the method 550 moves to step560.

Step 560 considers if there is a match in scenario 3.2. This means oneor more combination of available resources match phrase(s) from thepossible phrase table. Other phrases are rejected. If there is a matchin scenario 3.2, the method 550 is in scenario 3.2 and otherwise, themethod 550 is in scenario 3.1.

It is noted that FIG. 5 shows but one example hierarchical analysismethod 550 that may be utilized in embodiments. Embodiments mayimplement any desired hierarchical analysis logic that is tailored tothe environments being simulated. Further, it is noted that the method550 may be used to generate a first phrase for an operation. Then, if auser wants to determine phrases for other tasks that are part of theoperation, a mapping, e.g., as shown in the table 782 of FIG. 7G,between a generated phrase (e.g., generated using the method 550) andphrases that occur before and after the generated phrase are used. Forexample, in such an embodiment, the phrase generated using the method550 (parts of which are in columns A-D) is looked up in the table 782and the additional phrases that are before and after the generatedphrase are indicated in columns J-M and N-Q, respectively. As such, thetable 782 includes a mapping between a generated phrase (shown incolumns A-D) and a phrase that occurs before (shown in columns J-M) anda phrase that occurs after (shown in columns N-Q). It is noted that thetable 782 illustrates but a portion of one example phrase table that maybe used in embodiments and various tables may be utilized that areparticular to user desired functionality and the applications andcontexts for which embodiments are being implemented.

As noted above, in embodiments, e.g., the system 440, after determininga valid phrase, the phrase is used to determine inputs, e.g., grasptarget 453, vision target 454, and vision acuity 455, for a posturingengine, e.g., posturing engine 446. Determining the posture engineinputs from the phrase, according to an embodiment, includes identifyingthe object to interact with. In an embodiment, the object to interactwith is called the grasp targeted object. The grasp target object is notnecessarily the “what” item of the phrase. For example, in the phrase“Get the assembly from rack”, the “What” item is the “Assembly”, whichis the targeted object. However, in the phrase Get the assembly with alift assist, the “What” is still the object to “Get.” However, since theget is done by the lift assist, the manikin will interact with the“With” (lift assist). An embodiment identifies the object to interactwith using a set of targeted object rules. The target object rules aremapping between phrase elements and target objects. Specifically, therules indicate the particular target object given particular phrases orcharacteristics of phrases. In an example embodiment the rules indicatethat if the phrase has a “with” preposition and the With or Where 452 isa resource, the targeted object is the With or Where 452 object of thephrase and, else, the targeted object is the What 451 object.

In a similar way to the grasp targeted object definition, the phrase isused to define the vision target. However, the challenge is different.The vision target may be different from the grasping target. Forexample, when grabbing an assembly with a lifting device, the grasptarget is the lifting device handle, but the vision target is theassembly. Embodiments use rules/mapping that embody logic that uses theelements of the phrase, with an emphasis on the action and thepreposition, to define the vision target.

Further, an embodiment defines a more accurate location in relation tothe vision target. This precision is influenced by the action to beperformed in the simulation. For example, getting a hand tool, like apower screwdriver, requires looking at the tool in general. However,using the hand tool requires looking at the tip of the tool. In anotherexample, carrying the power screwdriver does not require looking at it.Embodiments can utilize a mapping that considers these nuances andappropriately determines the vision target given the task

A worker task can be a task where each hand is doing a different task.For example, a task can include the left hand picking up bolts from abin while the right hand gets a screwdriver. In an embodiment, twophrases are identified for tasks where each hand is doing somethingdifferent. To avoid conflict when two phrases are defined for looking atdifferent targets, another logic is applied to prioritize the mostimportant hand action. An example of this logic is shown in the table660 of FIG. 6 . In the example table 660, column 661 indicates the tasksa left hand is doing, e.g., aligning, assembling, bending etc., and therow 662 indicates the tasks a right hand is going, e.g., aligning,assembling, bending, etc. The intersections of the column 661 and row662 indicate the most important hand action and the vision priority. Forinstance, if the left hand action is connecting 663 and the right handaction is cleaning 664, the prioritized task is the connecting 665. Itis noted that the table 660 illustrates but one example prioritizationlogic that may be used in embodiments and any variety of prioritizationschemes can be utilized that are particular to user desiredfunctionality and the applications and contexts for which embodimentsare being implemented.

Once the vision target is defined, an embodiment also determines visionacuity. Orienting the manikin line of sight on the visual targetoptimizes the eye to distinguish color and shape. However, it can createan ergonomically undesirable posture. In an embodiment, the posturingengine considers required vision acuity in determining the posture.Thus, an embodiment also determines a vision acuity as a posture engineinput. An example embodiment uses three different levels of visionacuity, precision, in field of view, and not required. Further, anembodiment determines the vision acuity based on what acuity is neededto perform the action in the phrase. In the example of a getting a powerscrewdriver, this task requires a within field of view while in theexample of using a power screwdriver, a precision view acuity level isneeded.

FIGS. 7A-G illustrate steps of using an embodiment to posture a manikin.

The example illustrated in FIGS. 7A-G involves simulating a task of anoperation, where a part, bolt 771, is assigned to the operation to beadded to the product 772. In this example, a power screwdriver 773 iswithin the working environment 770.

An embodiment checks several rules and then adds object(s) to a list ofcandidate items for the phrase. First, such an embodiment determines ifthe operation inherits a part or an assembly from a previous operation.FIG. 7B shows that the part 772 in the environment 770 is inherited froma previous operation.

Next, the embodiment analyzes data regarding the operation anddetermines the operation involves providing a new part 771. In thiscase, BoltM6-14 771 is assigned to the task and, as such, the task isconsidered to include providing a new part. This is determined in anembodiment by examining process planning data. The interface 774 in FIG.7C is an example user interface 774 that embodies the process planningdata where the bolt 771 assignment 775 is shown.

To continue, the embodiment considers if the concerned operation isproviding (resource assignations) objects from a resource tree. In thepresent use case, as shown in the interface 776 of FIG. 7D, an assemblytray 777 and the power tool 778 are assigned to the task. In thisexample embodiment, if the resource object is not identified by arelevant category, it is not added to the candidate list of words forthe phrase.

The embodiment also considers any surrounding resources in collisionwith a vertical cylinder 779 (shown in FIG. 7E) centered at theoperation position (if defined), else around resource position (ifdefined). According to an embodiment, an operation position is a userdeclared position of the product buildup to be applied when theoperation is selected. A resource position is similar to an operationposition, but a resource position is applied on an assigned resource. Inthis example, as shown in the environment 770 in FIG. 7E, the verticalcylinder 779 is defined around the center of the product buildup 772.Once again, if the resource object is not identified by a relevantcategory, it is not added to the candidate list.

After performing the foregoing checks described in relation to FIGS.7A-E, a list of candidate phrase words is developed. FIG. 7F shows anexample candidate list 781 in the interface 780. The list 781 can beprovided to a user for modifying the phrase manually, but, in thisexample, it is first used by the system to determine the task to beexecuted by the manikin.

The example illustrated using FIGS. 7A-G relies on a table of allpossible phrases. In this example, the table (of which an exampleportion 782 is shown in FIG. 8G) includes 667 phrases. Each phrase iscomposed of an “action verb”, a “what” object, a “preposition,” and a“with or where” object. This table 782 format enables expansion to addmore phrases as needed depending on the application of embodiments,e.g., industries to cover, or purpose, such as manufacturing,maintenance, etc., for which embodiments are being utilized.

In this example, the list of candidate phrase components is used tofilter the table 782 to only valid phrases. After this, such anembodiment filters the table 782 to include a minimal quantity ofphrases. In this example, a filtering method, e.g., the method 550, isused to filter the table. Embodiments can use different strategies forfiltering depending upon the presence or absence of an object's categorywithin the list of candidate items. In an embodiment the phrasesresulting from the filtering process are then prioritized per resourcecategory. A phrase utilizing a hand tool is prioritized over oneutilizing an industrial device. According to an embodiment, the priorityorder from the highest priority to the lowest is as follows: hand tool,industrial device, logistics, storage, furniture, computer equipment,and others.

First, an embodiment categorizes the candidate phrase objects in threetypes: provided (a new part implemented by the operation), assembly (theresult of the previous operation), and resources (objects needed tomanufacture the product). Second, the embodiment classifies the itemsdefined as a resource, into sub categories: hand tool, industrial device(such as lift assistant, machine, robot, jig, etc.), logistics(conveyor, automated guided vehicle, cart), storage, and furniture.Validating a manikin interacting with a hand tool has more value thanwith a furniture. Third, the hierarchical analysis guides the tablereduction as a function of the candidate item possible combinationsaccording to the following combination order (deducted frommanufacturing premises): (1) high value assigned resource and providedobject, (2) high value resource in proximity and provided object, (3)high value assigned resource and assembly, (4) high value resource inproximity and assembly, (5) provided object and assembly, (6) assemblyonly, (7) provided object only, (8) lower value resource (assigned ornot), etc.

In the example of FIGS. 7A-G the power screwdriver 773 is a high valueresource. Thus, all phrases that do not have a power screwdriver in the“What” or in the “With or Where” are removed. The concerned operationprovides a new part (bolt 771) to be assembled. This combination (highvalue assigned resource (screwdriver 773) and provided new object (bolt771)) leads to the removal of all the remaining phrases that do notinclude a “Provided” item. The leaves such an embodiment with thephrases shown below in Table 1.

TABLE 1 Screw Provided with Ergo_PowerScrewdriver Tighten Provided withErgo_PowerScrewdriver Unscrew Provided with Ergo_PowerScrewdriver

After obtaining the candidate phrases in Table 1, another filter basedon the action verb is applied. The example illustrated across FIGS. 7A-Grelates to manufacturing and, thus, in this manufacturing domain themain action determination prioritizes actions of assembling overdisassembling. In this example, the action “screw” is prioritized over“Tighten” since the tool is a screwdriver. Tighten would have beenprioritized if the tool was a nut runner. This filter yields thecandidate phrase shown below in Table 2.

TABLE 2 Screw Provided with Ergo_PowerScrewdriver

In turn, the example of FIGS. 7A-G continues and determines the postureengine inputs using a mapping between the phrase in Table 2 and theposture engine inputs. In this example, there are posture engine inputsassociated to that specific line in the phrase table 782 and the systemsends the input to the posturing engine. Here, the power screwdriver issent as the hand target object, the tool tip is sent as the visiontarget, and within field of view is sent as the vision accuracy needed.

Embodiments have numerous advantages compared to existing posturedetermination techniques. Embodiments, through the development and useof the phrase, are significantly more efficient. Further, embodimentscan also be leveraged for automatic object positioning. With theaddition of more 3D object characteristics, embodiments can determineposition automatically for an object mentioned in the phrase. Forexample, if the phrase is “get bolt from bin,” an embodiment can setthat the bolt is automatically moved into the bin so the user does nothave to position the bolt. Such functionality combined with embodimentsand posturing engines, significantly reduces the manual work required byusers and allows users to easily assess ergonomics.

Combined with the Smart Posturing Engine™ (SPE™) technology (4, 5, 6),which generates posture automatically in a 3D environment, Ergo4All™technology (7) analyses the potential risk of developing musculoskeletaldisorder (MSD) by workers. Embodiments provide guidance to user onchanges to environments, e.g., workstations, that will lower ergonomicrisks. Further, embodiments can provide inputs to the posturing engineand allow the user to assess ergonomics in one click.

Advantageously, embodiments reuse process planning information. Duringan engineering phase, before an ergonomics assessment, engineers definethe operations, assign the parts and resources to those operations, andset the flow between operations, in order to do product build-up reviewand line balancing. Since this information already exists, it can beused by embodiments to predict the user intention regarding ergonomicsand consequently reduce the workload.

Further, embodiments speed up manikin posturing. Embodiments generate asimple and logical phrase to determine the manikin task postureautomatically. This avoids having to manually specify where the hands,the feet, and the vision for a manikin in a simulation should be.

Embodiments also standardize results between users. By using embodimentsto determine the manikin posture, the posture is not subject toindividual interpretation. Thus, the resulting ergonomic assessment orother such simulation results generated using embodiments is not userdependent.

Unlike existing methods, embodiments provide functionality toautomatically generate a phrase based upon environment data. Incontrast, existing methods require a user to select elements via aseries of menus. Embodiments provide a unique method that automaticallyuses process planning data to determine manikin posture. Advantageouslyembodiments automatically determine the inputs for a posture engine.

Computer Support

FIG. 8 is a simplified block diagram of a computer-based system 880 thatmay be used to determine position of a manikin according to any varietyof the embodiments of the present invention described herein. The system880 comprises a bus 883. The bus 883 serves as an interconnect betweenthe various components of the system 880. Connected to the bus 883 is aninput/output device interface 886 for connecting various input andoutput devices such as a keyboard, mouse, display, speakers, etc. to thesystem 880. A central processing unit (CPU) 882 is connected to the bus883 and provides for the execution of computer instructions. Memory 885provides volatile storage for data used for carrying out computerinstructions. In particular, memory 885 and storage 884 hold computerinstructions and data (databases, tables, etc.) for carrying out methods100, 440, 550 of FIGS. 1, 4, and 5 and supporting corresponding userinterfaces described above. Storage 884 provides non-volatile storagefor software instructions, such as an operating system (not shown). Thesystem 880 also comprises a network interface 881 for connecting to anyvariety of networks known in the art, including wide area networks(WANs) and local area networks (LANs).

It should be understood that the example embodiments described hereinmay be implemented in many different ways. In some instances, thevarious methods and machines described herein may each be implemented bya physical, virtual, or hybrid general purpose computer, such as thecomputer system 880, or a computer network environment such as thecomputer environment 990, described herein below in relation to FIG. 9 .The computer system 880 may be transformed into the machines thatexecute the methods (e.g., 100, 440, 550) and techniques describedherein, for example, by loading software instructions into either memory885 or non-volatile storage 884 for execution by the CPU 882. One ofordinary skill in the art should further understand that the system 880and its various components may be configured to carry out anyembodiments or combination of embodiments of the present inventiondescribed herein. Further, the system 880 may implement the variousembodiments described herein utilizing any combination of hardware,software, and firmware modules operatively coupled, internally, orexternally, to the system 880.

FIG. 9 illustrates a computer network environment 990 in which anembodiment of the present invention may be implemented. In the computernetwork environment 990, the server 991 is linked through thecommunications network 992 to the clients 993 a-n. The environment 990may be used to allow the clients 993 a-n, alone or in combination withthe server 991, to execute any of the embodiments described herein. Fornon-limiting example, computer network environment 990 provides cloudcomputing embodiments, software as a service (SAAS) embodiments, and thelike.

Embodiments or aspects thereof may be implemented in the form ofhardware, firmware, or software. If implemented in software, thesoftware may be stored on any non-transient computer readable mediumthat is configured to enable a processor to load the software or subsetsof instructions thereof. The processor then executes the instructionsand is configured to operate or cause an apparatus to operate in amanner as described herein.

Further, firmware, software, routines, or instructions may be describedherein as performing certain actions and/or functions of the dataprocessors. However, it should be appreciated that such descriptionscontained herein are merely for convenience and that such actions infact result from computing devices, processors, controllers, or otherdevices executing the firmware, software, routines, instructions, etc.

It should be understood that the flow diagrams, block diagrams, andnetwork diagrams may include more or fewer elements, be arrangeddifferently, or be represented differently. But it further should beunderstood that certain implementations may dictate the block andnetwork diagrams and the number of block and network diagramsillustrating the execution of the embodiments be implemented in aparticular way.

Accordingly, further embodiments may also be implemented in a variety ofcomputer architectures, physical, virtual, cloud computers, and/or somecombination thereof, and thus, the data processors described herein areintended for purposes of illustration only and not as a limitation ofthe embodiments.

The teachings of all patents, published applications and referencescited herein are incorporated by reference in their entirety.

While example embodiments have been particularly shown and described, itwill be understood by those skilled in the art that various changes inform and details may be made therein without departing from the scope ofthe embodiments encompassed by the appended claims.

REFERENCES

-   (1) Cort J A, Devries D. Accuracy of Postures Predicted Using a    Digital Human Model During Four Manual Exertion Tasks, and    Implications for Ergonomic Assessments. IISE Transactions on    Occupational Ergonomics and Human Factors. 2019; 7(1):43-58.-   (2) Hanson L, Högberg D, Carlson J S, Bohlin R, Brolin E, Delfs N,    et al. IMMA-Intelligently moving manikins in automotive    applications. Third International Summit on Human Simulation    (ISHS2014)2014.-   (3) Baerlocher, P. Inverse kinematics techniques for the interactive    posture control of articulated figures [Ph. D. thesis]. Ecole    Polytechnique Federale de Lausanne (EPFL)Stephens A, Jones M.    Workplace methods and use of digital human models. Handbook of    DIGITAL HUMAN MODELING, USA: Taylor and Francis. 2009:6.1-6.11.-   (4) Lemieux P-O, Barré A, Hagemeister N, Aissaoui R. Degrees of    freedom coupling adapted to the upper limb of a digital human model.    International Journal of Human Factors Modelling and Simulation.    2017; 5(4):314-37.-   (5) Lemieux P, Cauffiez M, Barré A, Hagemeister N, Aissaoui R. A    visual acuity constraint for digital human modeling. Conference    proceedings 4th. 2016.-   (6) Zeighami A, Lemieux P, Charland J, Hagemeister N, Aissaoui A.    Stepping behavior for stability control of a digital human model.    ISB/ASB. 2019.-   (7) Bourret Q, Charland J, Imbeau D, Brouillette D, Djire J-B.    Ergo4All: An Ergonomic Guidance Tool for Non-ergonomist. N. L. Black    et al. (Eds.): IEA 2021, LNNS 221, pp. 382-390, 2021.-   (8) Perez, J. and Neumann, W. P. (2015). Ergonomists' and Engineers'    views on the utility of virtual Human Factors Tools. Human Factors    and Ergonomics in Manufacturing & Service Industries, 25(3),    279-293.

What is claimed is:
 1. A computer-implemented method of determining aposture for a manikin in a simulation of a real-world environment, themethod comprising: receiving environment data; automatically generatinga phrase by performing a hierarchical analysis using the receivedenvironment data, wherein the generated phrase describes a task, to besimulated, performed by a manikin in an environment; determining one ormore posture engine inputs based on the generated phrase; andautomatically determining a posture for the manikin in a simulation ofthe manikin performing the task in the environment based on thedetermined one or more posture engine inputs.
 2. The method of claim 1wherein the environment data includes at least one of: one or moreoperations; a sequence of tasks comprising an operation; one or moreparts assigned to an operation; one or more resources assigned to anoperation; a sequence of operations; one or more objects in theenvironment; position of the one or more objects in the environment;context of the task; and task validation premises.
 3. The method ofclaim 1 wherein the phrase is comprised of: (i) a verb componentindicating an action of the task, (ii) a what component indicating whatthe verb component pertains to, (iii) a preposition component, and (iv)a with or where component indicating an object used for the action orwhere the action is performed.
 4. The method of claim 1 whereinautomatically generating a phrase by performing a hierarchical analysisusing the received environment data comprises: determining candidatephrases based on the environment data; and hierarchically filtering thecandidate phrases based on the environment data to generate the phrase.5. The method of claim 4 wherein hierarchically filtering the candidatephrases comprises: first, eliminating invalid phrases from thedetermined candidate phrases; and second, eliminating phrases with aninvalid interaction object.
 6. The method of claim 1 further comprising:prior to determining the one or more posture engine inputs, receiving anindication of user approval of the phrase.
 7. The method of claim 1wherein the one or more posture engine inputs include at least one of:grasp target, vision target, vision acuity for the manikin performingthe task, and an indication if object weight is considered.
 8. Themethod of claim 1 wherein determining the one or more posture engineinputs based on the generated phrase comprises: searching a mappingusing components of the generated phrase, wherein results of thesearching indicate the one or more posture engine inputs.
 9. The methodof claim 8 wherein the mapping indicates respective posture engineinputs corresponding to respective phrase components.
 10. The method ofclaim 1 wherein automatically determining the posture comprises:providing the determined posture engine inputs to a posturing engineconfigured to determine the posture based upon the provided postureengine inputs; and receiving the posture from the posture engine. 11.The method of claim 1 further comprising: simulating the manikinperforming the task in the environment using the determined posture. 12.The method of claim 11 further comprising: determining a change to aworkstation for the manikin performing the task based on results of thesimulation.
 13. A system for determining a posture for a manikin in asimulation of a real-world environment, the system comprising: aprocessor; and a memory with computer code instructions stored thereon,the processor and the memory, with the computer code instructions, beingconfigured to cause the system to: receive environment data;automatically generate a phrase by performing a hierarchical analysisusing the received environment data, wherein the generated phrasedescribes a task, to be simulated, performed by a manikin in anenvironment; determine one or more posture engine inputs based on thegenerated phrase; and automatically determine a posture for the manikinin a simulation of the manikin performing the task in the environmentbased on the determined one or more posture engine inputs.
 14. Thesystem of claim 13 wherein the phrase is comprised of: (i) a verbcomponent indicating an action of the task, (ii) a what componentindicating what the verb component pertains to, (iii) a prepositioncomponent, and (iv) a with or where component indicating an object usedfor the action or where the action is performed.
 15. The system of claim1 wherein, in automatically generating a phrase by performing ahierarchical analysis using the received environment data, the processorand the memory, with the computer code instructions, are furtherconfigured to cause the system to: determine candidate phrases based onthe environment data; and hierarchically filter the candidate phrasesbased on the environment data to generate the phrase.
 16. The system ofclaim 15 wherein, in hierarchically filtering the candidate phrases, theprocessor and the memory, with the computer code instructions, arefurther configured to cause the system to: first, eliminate invalidphrases from the determined candidate phrases; and second, eliminatephrases with an invalid interaction object.
 17. The system of claim 13wherein the processor and the memory, with the computer codeinstructions, are further configured to cause the system to: prior todetermining the one or more posture engine inputs, receive an indicationof user approval of the phrase.
 18. The system of claim 13 wherein, indetermining the one or more posture engine inputs based on the generatedphrase, the processor and the memory, with the computer codeinstructions, are further configured to cause the system to: search amapping using components of the generated phrase, wherein results of thesearching indicate the one or more posture engine inputs and the mappingindicates respective posture engine inputs corresponding to respectivephrase components.
 19. The system of claim 13 wherein, in automaticallydetermining the posture, the processor and the memory, with the computercode instructions, are further configured to cause the system to:provide the determined posture engine inputs to a posturing engineconfigured to determine the posture based upon the provided postureengine inputs; and receive the posture from the posture engine.
 20. Anon-transitory computer program product for determining a posture for amanikin in a simulation of a real-world environment, the computerprogram product executed by a server in communication across a networkwith one or more clients and comprising: a computer readable medium, thecomputer readable medium comprising program instructions which, whenexecuted by a processor, causes the processor to: receive environmentdata; automatically generate a phrase by performing a hierarchicalanalysis using the received environment data, wherein the generatedphrase describes a task, to be simulated, performed by a manikin in anenvironment; determine one or more posture engine inputs based on thegenerated phrase; and automatically determine a posture for the manikinin a simulation of the manikin performing the task in the environmentbased on the determined one or more posture engine inputs.